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[ABSTRACT] 



[Abstract! 

A method for enabling a mobile terminal equipped with a camera to store 
document information. An image of a document picked up by the camera is dis 
5 played. In response to a document "RECOGNIZE" key input, a character imag 
e is recognized from the displayed document image and the character image is co 
nverted into character data. The character data is displayed on the first display 
area, and SAVE items are displayed on the second display area. A SAVE item i 
s selected from the displayed SAVE items and character data associated with the 
10 selected SAVE item is selected, such that the selected SAVE item and the charact 
er data associated with the selected SAVE item can be displayed on the third disp 
lay area. When a "CORRECT" key is inputted, candidate characters associated 
with an erroneously recognized character are displayed on the third display area. 
After the erroneously recognized character is corrected or replaced with a sele 
1 5 cted candidate character, a result of the correction is displayed on the third displa 
y area. When a "CONFIRM" key is inputted, the SAVE item and character data 
displayed on the third display area are stored. 



[Representative Drawing] 



FIG. 1 



20 



[Keywords] 



Character recognition, Image, Card, Phone book 



-1- 




2002-55148 

[SPECIFICATION] 

[Title of Invention] 

DEVICE AND METHOD FOR STORING DOCUMENT INFORMATIO 
N USING CAMERA 

5 [Brief Description of the Drawings] 

FIG. 1 is a block diagram illustrating the exemplary configuration of an a 
pparatus for recognizing characters in accordance with the present invention; 

FIG. 2 is a flow chart illustrating a character recognition method in accord 
ance with the first embodiment of the present invention; 
10 FIG. 3 is a detailed flow chart illustrating a document pick-up process sho 

wn in FIG. 2; 

FIG. 4 is a flow chart illustrating character recognition and SAVE item sel 
ection processes shown in FIG. 2; 

FIG. 5 is a flow chart illustrating an error correction process shown in FI 

15 G. 2; 

FIGS. 6A to 6E are views illustrating display screen states of a display uni 
t in a document pickup process; 

FIGS. 7 A and 7B are views illustrating display screen states of the display 
unit in the character recognition and SAVE item selection processes; 
20 FIGS. 8A to 8D are views illustrating display screen states of the display 

unit in the error correction process; 

FIGS. 9 A and 9B are views illustrating display screen states of the display 
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unit after the error correction process; 

FIG. 10 is a flow chart illustrating a document recognition process in acco 
rdance with the second embodiment of the present invention; 

FIG. 1 1 is a flow chart illustrating a document pickup process shown in FI 

G. 10; 

FIG. 12 is a flow chart illustrating character recognition, SAVE item selec 
tion and storage processes shown in FIG. 10; 

FIG. 13 is a flow chart illustrating the SAVE item selection process show 
nin FIG. 12; and 

FIGS. 14A to 14D are flow charts illustrating the error correction process 
shown in FIG. 10. 

[Detailed Description of the Invention] 
[Objects of the Invention] 

[Technical Fields of the Invention and Related Art] 

1 5 The present invention relates to a character recognition apparatus and met 

hod, and more particularly to an apparatus and method for recognizing a characte 
r image from an image or picture screen. 

Mobile terminals have recently developed into a structure capable of trans 
mitting high-speed data. In particular, mobile communication networks based o 
20 n an IMT-2000 (International Mobile Telecommunication-2000) standard can im 
plement high-speed data communications using small-sized mobile terminals. 
Data processable in the mobile terminals for performing the data communication 
s can be packet data and image or picture data. 
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As a limited keypad is used when the above-described mobile terminals re 
ceive input information, a character input method is complex. When the mobile 
terminals use an input unit based on a soft keyboard, a character input rate is slo 
w and a character input method is very complex. Thus, a character recognition 
5 device and/or speech recognition device can be used to address a drawback of th 
e soft keyboard. There is a problem in that character recognition and input rates 
are slow where a handwritten character recognition device is used. Similarly, w 
here the speech recognition device is used, there is a problem in that only a limite 
d number of words can be recognized. For this reason, a keyboard input unit in 
10 eluding a separate hardware device can be used to input characters. However, t 
he above-described method has a problem that an additional device for inputting 
characters must be provided in the mobile terminals. 

It is a trend that an image processing function is added to the mobile termi 
nal and hence the mobile terminal can have a composite function. In this case, 

15 an image processing device of the mobile terminal includes a camera for picking 
up an image and a display unit for displaying signals of the image picked up by t 
he camera. Here, the camera can use a CCD (Charge Coupled Device) image s 
ensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor, a 
nd the display unit can use an LCD (Liquid Crystal Display). As small-sized ca 

20 mera devices have been developed, image pickup devices are currently miniaturi 
zed. In this case, the mobile terminal can perform an image pickup operation, d 
isplay a moving picture and a still picture on a screen, and transmit picked-up im 
ages. However, the mobile terminal equipped with the camera performs only fu 
nctions of picking up, storing, managing and transmitting images or pictures. 
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The mobile terminal can be a mobile phone or PDA (Personal Digital Assi 
stant). In a character input method of the PDA, characters based on a soft keyp 
ad are inputted with a stylus pen or characters are inputted through handwritten c 
haracter recognition. However, where a large number of characters are inputted 
5 by the conventional character input methods, users feel inconvenience because 
of a slow process rate and a complex work. In particular, when contents of a ca 
rd bearing a person's name and other information are inputted into the PDA, muc 
h time and effort are required. Thus, a method capable of improving current in 
put methods or facilitating the convenience of the users is seriously required. T 
10 he present invention can pick up a document using a camera with which a termin 
al device is equipped, recognize an image of the picked-up document in the form 
of characters, and store the recognized image, thereby improving a character in 
put function of the mobile terminal. 



ems, and it is one object of the present invention to provide an apparatus and met 
hod, which can pick up a document using a camera with which a terminal device 
is equipped, recognize an image of the picked-up document in the form of charac 
ters, and store the recognized image, thereby improving a character input functio 
20 n of the mobile terminal. 



[Technical Subjects to be solved by the Invention] 



15 



Therefore, the present invention has been made in view of the above probl 



It is another object of the present invention to provide an apparatus and m 
ethod, which can pick up a document using a camera with which a terminal devic 
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e is equipped, recognize an image of the picked-up document in the form of char 
acters, and correct erroneously recognized characters. 

It is another object of the present invention to provide an apparatus and m 
ethod, which can pick up a document using a camera with which a terminal devic 
5 e is equipped, recognize characters from an image of the picked-up document, an 
d correct erroneously recognized characters using a candidate character table in a 
correction process. 

It is another object of the present invention to provide an apparatus and m 
ethod, which can pick up a document using a camera and speech recognizer with 
10 which a terminal is equipped, recognize characters from an image of the picked- 
up document, and correct erroneously recognized characters using a speech reco 
gnizer in a correction process. 

It is another object of the present invention to provide an apparatus and m 
ethod, which can pick up a document using a camera and handwritten character r 
1 5 ecognizer with which a terminal is equipped, recognize characters from an image 
of the picked-up document, and correct erroneously recognized characters in a c 
orrection process by recognizing handwritten characters inputted by a user. 

It is another object of the present invention to provide an apparatus and m 
ethod, which can pick up a document using a camera with which a terminal is eq 
20 uipped, recognize an image of the picked-up document as characters, and correct 
erroneous characters using a soft keypad in a correction process. 
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It is yet another object of the present invention to provide an apparatus an 
d method, which can pick up a document containing phone book information usi 
ng a camera with which a mobile communication terminal is equipped, and recog 
nize and save the phone book information contained within an image of the picke 
5 d-up document. 

[Description of the Preferred Embodiments] 

Now, preferred embodiments of the present invention will be described in 
detail with reference to the annexed drawings. In the drawings, the same or sim 
ilar elements are denoted by the same reference numerals even though they are d 
10 epicted in different drawings. 

Hereinafter, specific details such as a card and PDA are provided so that t 
he present invention can be better understood. Those skilled in the art will appr 
eciate that the present invention can be easily implemented, without the specific 
details. In the embodiment of the present invention, it is assumed that a docum 
15 ent pick-up by a camera is a card bearing a person's name and a mobile terminal 
device is the PDA. 

When characters are inputted into the mobile terminal, a user's ability to i 
nput characters can be improved, a user manipulation of an input unit can be sim 
plified, characters erroneously recognized in a character recognition process can 
20 be easily corrected by speech recognition, and a large amount of text can be input 
ted. 
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To do this, an image pre-processing algorithm is developed to perform a c 
haracter recognition process in an internal or external camera coupled to the mob 
ile terminal, a character recognition engine for recognizing an image picked up b 
y the internal or external camera is developed, and a speech recognition engine f 
5 or correcting erroneously recognized characters is developed, a user interface for 
inputting characters according to character and speech recognitions is implemen 
ted, and a user interface for correcting erroneously recognized characters on the 
basis of the speech recognition is implemented. 

To implement the above-described functions, it is preferable that a camera 
capable of carrying out fine focus adjustment is used in the terminal device. T 
he reason why the fine focus adjustment is used is to enhance the resolution of a 
document image to be recognized. In order for the camera to improve an image 
pre-processing function, image quality must be excellent, and fine focus adjust 
ment must be possible, and a preview screen must be able to be used so that the u 
ser can determine an optimum focus state. When an object's position is adjuste 
d on the screen, a screen update speed must be fast and the object must be focuse 
d at the center of the screen. 

As described above, an image pre-processing function for character recog 
nition needs the support of hardware and software specifications. The hardware 
20 specification must support a fine focus adjustment function for a picked-up ima 
ge, ensure the minimum display rate of 12 fps so that an optimum focus state can 
be confirmed in a focus adjustment process, and ensure an excellent lens necess 
ary for acquiring the best image quality for character recognition. A software p 
re-process must be able to remove image distortion from an original camera imag 
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e obtained via a pinhole lens, remove distortion caused by a focus mismatch of a 
n image picked up in a near view field from the image, determine whether charac 
ter size and focus adjustments are appropriate for character recognition, remove i 
mage distortion caused by non- vertical projection for an object from the image, a 
5 nd binarize character data being an object under various illumination conditions. 

As described above, the character recognition function is needed to recog 
nize an image of a document picked up by the camera. For the character recogn 
ition, an engine for recognizing optical characters must be developed, an amount 
of processable data associated with the engine must be less than a predetermined 

10 amount of processable data (e.g., 5 Mbytes), various fonts of printed English lett 
ers, Korean characters and digits must be able to be recognized, and a minimum r 
ecognition percentage per character must be 80%. Further, it is preferable that 
a speech recognition module be provided in the terminal device so that an errone 
ous character can be corrected in an error correction process. Furthermore, a us 

15 er interface for a text input by the character recognition and speech recognition 
must be implemented. 

We assume that the terminal device in accordance with an embodiment of 
the present invention is a PDA (Personal Digital Assistant), and a picked-up docu 
ment is a card bearing a person's name and other information. An image of the 
20 card is picked up, the picked-up image is pre-processed, a character image is extr 
acted from the pre-processed image, the extracted character image is recognized, 
the recognized character image is converted into character data, erroneously reco 
gnized character data is corrected, and the corrected character data is stored in a 
phone book. This example will now be described in detail. 
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In accordance with the embodiment of the present invention, the followin 
g method can easily input, into the PDA, a document having a large amount of in 
formation such as the card bearing a person's name and other information using 
various input units (e.g., a character recognizer, a speech recognizer, a pen and a 
5 keyboard). 

First, an image of the card or document is picked up using a camera embe 
dded in the PDA, a character image contained within the picked-up image is pre- 
processed by a pre-processor so that a clear character image can be produced, the 
pre-processed character image is recognized by the character recognizer, and th 
10 e recognized character image is converted into character data. Erroneously reco 
gnized character data is corrected using various devices such as a stylus pen, a sp 
eech recognizer, a handwritten character recognizer, a soft keypad, etc. and the c 
haracter data is stored in a desired storage area of a database. 

FIG. 1 is a block diagram illustrating the exemplary configuration of a mo 
1 5 bile terminal for recognizing a character image from an image screen in accordan 
ce with the present invention. 

Referring to FIG. 1, the mobile terminal includes a controller 101 for cont 
rolling an overall operation of the mobile terminal that recognizes a document. 
A memory 1 03 carries out a function of storing a program for controlling the ope 
20 ration of the mobile terminal or temporarily storing data generated while the prog 
ram is executed. 
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A camera 107 carries out a function of picking up an image of the docume 
nt. Here, the document can be a card bearing a person's name and other inform 
ation. The camera 107 can carry out a pre-processing function. In other word 
s, the camera 107 can adjust a focus and focal distance for an object, and enhanc 
5 e quality of the picked-up image. An image processor 109 can carry out functio 
ns of converting the picked-up image into digital data and compressing the digita 
1 data. The image processor 109 can use an image processor disclosed in Korea 
n Patent Application No. 2002-22844 previously filed by the applicant of the pre 
sent invention. 

1 0 An audio processor 111 processes a speech signal used for correcting an e 

rroneous character generated while the program is executed, and processes a spe 
ech signal used for displaying a result of the execution and guidance while the pr 
ogram is executed. An input unit 1 13 as a touch screen module can be unified 
with a display unit 115. 

1 5 The input unit 113 allows a user to input a desired character and a functio 

n key using a stylus pen. The input unit 113 includes a "SHOOT" key, a "REC 
OGNIZE" key, a "CONFIRM" key, a "CORRECT" key, an "END" key, an "INS 
ERT" key, a "CANCEL" key, etc. The "SHOOT" key is used for picking up an 
image, displaying the picked-up image and storing the picked-up image display 

20 ed on a screen. The "RECOGNIZE" key is used for recognizing a character im 
age from a currently displayed image screen. Where recognized documents hav 
e different specific formats, different document recognition keys can be provided 
For example, where a document is a card bearing a person's name and other i 
nformation, the document information can configure a phone book of the mobile 
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terminal. In this case, a card "RECOGNIZE" key is provided on the input unit 
113. If common information items recorded on cards are selectively stored in a 
table, the mobile terminal's phone book can be easily created. The "CONFIRM 
" key is used for registering character data of a selected item. The "CORRECT 
5 " key is used for correcting character data of a selected item. The "INSERT" ke 
y is used for inserting a character in a selected text position at which a cursor is p 
laced. In other words, where at least one character is omitted from a text of a c 
haracter recognition result, a new character can be inserted before a cursor positi 
on. The "CANCEL" key is used for canceling character data of a selected item. 
10 The "END" key is used for completing a current operation. 

A key input unit 105 includes function keys for setting various functions o 
f the mobile terminal. Here, the function keys capable of being arranged on the 
key input unit 105 include a speech "RECOGNIZE" key for driving the speech r 
ecognizer 129, a focus and focal distance adjustment key for controlling a pre-pr 

15 ocessing operation of the camera 107 and a "SHOOT" key for storing a preview i 
mage outputted by the camera 107. Of course, the keys arranged on the key inp 
ut unit 105 can be provided in the input unit 113. In the embodiment of the pres 
ent invention, we assume that all function keys are arranged on the input unit 113 
for convenience of explanation. Here, the camera 107, the input unit 113, the 

20 audio processor 111 and the key input unit 105 can operate as an input device, re 
spectively. 

The display unit 115 carries out a function of displaying a result of a chara 
cter recognition process performed in accordance with the embodiment of the pre 
sent invention. In other words, the display unit 115 displays an image picked u 
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p by the camera 107 as a preview screen and displays the result of the character r 
ecognition in a character recognition mode. The display unit 115 includes a dis 
play area capable of displaying a result of an error correction process. The disp 
lay unit 115 includes the first display area 71, the second display area 75 and the 
third display area 73. The first display area 71 displays recognized character da 
ta, the third display area 73 displays character data associated with a selected SA 
VE item or candidate character data for the error correction process, and the seco 
nd display area 75 can include a display area for selectively displaying SAVE ite 
m information, handwritten characters inputted to correct an error and/or a soft k 
eypad for inputting desired characters using soft keys. A specific area for displa 
ying menu information for various command inputs in the character recognition 
process can be appropriately positioned in the first, second and third display area 
s 71, 75 and 73 in accordance with the embodiment of the present invention. 

When a character "RECOGNIZE" key is inputted from the input unit 113, 
a controller 101 drives a character recognizer 155. The character recognizer 1 
55 carries out a function of recognizing at least one character image from the inp 
ut image within a preview screen displayed on the display unit 115 and convertin 
g the recognized character image into character data. Further, recognized chara 
cter data is displayed on the first display area 71 of the display unit 115 under the 
control of the controller 101. Here, the character recognizer 155 can be config 
ured by a printed-character recognition module and a handwritten character reco 
gnition module. The printed-character recognition module can be used for reco 
gnizing a character image from the input image of the preview screen picked up 
by the camera 107, and the handwritten character recognition module can be use 
d for recognizing a handwritten character image inputted in the error correction p 
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rocess. The character recognizer 155 can include a module capable of converti 
ng soft key data inputted from the soft keypad into characters. 

The controller 101 drives a recognition error processor 157 when an error 
"CORRECT" key is inputted from the input unit 113. The recognition error pro 
5 cessor 157 corrects erroneous characters in the character recognition process by c 
orrecting or replacing the erroneous characters selected from the character data d 
isplayed on the first display area 71 with correction characters produced by the s 
peech recognizer 153 or the character recognizer 155. 

The controller 101 drives the speech recognizer 153 when the speech "RE 
10 COGNIZE" key is inputted in a state where the error "CORRECT" key is inputte 
d. The speech recognizer 153 recognizes a speech signal received from the aud 
io processor 111. The speech signal is inputted so that a desired item can be sel 
ected for error correction and an erroneous character associated with the selected 
item can be corrected. The speech recognizer 155 carries out a function of con 
15 verting the speech signal, inputted for correcting the erroneous character, into ch 
aracter data. Under the control of the controller 101, a speech synthesizer 151c 
arries out a function of synthesizing speech signals of character data as a result o 
f the recognition and outputting the synthesized speech signals in a speech output 
mode. 

20 A database 159 carries out a storage function so that a plurality of the reco 

gnized character data correspond to respective items. Here, where a document 
having the recognized character data is a card having a person's name and other i 
nformation, the database 159 can be a phone book memory or an address book m 
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emory. A user interface 161 carries out a function of interfacing user data coup 
led to the terminal device with the mobile terminal. 

As described above, the mobile terminal in accordance with the embodim 
ent of the present invention is configured by a camera module, an input module ( 
5 containing a touch screen), an audio module, a pre-processing module, a characte 
r recognition module, a recognition error correction module, a synthesis module, 
a user interface module, etc. The mobile terminal is operated by six processes o 
n a large scale. The six processes may be an image input process, an image pre- 
process, a character recognition process, a SAVE item selection process, an error 

10 correction process and a storage process. The processes are organically couple 
d with one another, and can be implemented by various methods. Major modul 
es used for the respective processes will be briefly described. The image input 
process is carried out by the camera module, the image pre-process is carried out 
by the pre-processing module, the character recognition process is carried out by 

1 5 the character recognition module and the speech recognition module, the SAVE i 
tern selection process is carried out by the speech recognition module and the inp 
ut module (containing a stylus pen), and the error correction process is carried ou 
t by the speech recognition module, the input module (containing the stylus pen), 
the handwritten character recognition module and a soft key recognition module 

20 , and the storage process is carried out by a database module. 

The document recognition process can be implemented by various method 
s. In accordance with the first embodiment of the present invention shown in FI 
G. 2, a document image is picked up, characters are recognized from the picked-u 
p document image, SAVE items associated with the recognized characters are sel 
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ected, a plurality of erroneously recognized character data corresponding to the s 
elected items are corrected, and a plurality of corrected character data is stored ac 
cording to a sequential process. Further, in accordance with the second embodi 
ment of the present invention, a document image is picked up, character data is r 
5 ecognized from a character image contained in the document image, an error corr 
ection item is selected, an erroneously recognized character is corrected, the corr 
ected character is stored, and the next error correction item is subsequently select 
ed. Hereinafter, it is explained that the error correction item selection and error 
correction processes are implemented by a document recognizer in accordance w 

10 ith the first embodiment of the present invention, and are implemented by the do 
cument recognizer and the speech recognizer in accordance with the second emb 
odiment of the present invention. However, the document recognition and corre 
ction can be carried out using the document recognizer and the speech recognizer 
in the first embodiment, and can be carried out using only the document recogni 

1 5 zer in the second embodiment. 

First, a document recognition method will be described in accordance wit 
h the first embodiment of the present invention. 

FIG. 2 is a flow chart illustrating the character recognition method in acco 
rdance with the first embodiment of the present invention. 

20 Referring to FIG. 2, the controller 101 enables a camera 107 to pick up a d 

ocument image so that the document image to be recognized can be produced at 
step 210. At this time, the document image picked up by the camera 107 is con 
verted into digital data by the image processor 109, and the digital data is display 
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ed on the display unit 115. When a still-picture capture command is issued in a 
state where the image is displayed on the display unit 115, the controller 101 ena 
bles the display unit 115 to display a still picture. An image displayed on the di 
splay unit 1 15 is stored in an image memory area of the memory 103. The imag 
5 e displayed on the display unit 1 1 5 can be a moving picture or character image da 
ta such as a card bearing a person's name and other information, etc. 

In the above-described state, the user of the terminal device inputs a key f 
or recognizing character images contained in the currently displayed image throu 
gh the input unit 113. Then, the controller 101 drives the character recognizer 1 

10 55 in response to an input of the character recognition key. Then, the character 
recognizer 155 recognizes a character image of the screen displayed on the displa 
y unit 115 and converts the recognized character image into character data. The 
controller 101 enables the first display area 71 of the display unit 1 15 to display 
the character data recognized by the character recognizer 123, and enables the se 

15 cond display area 75 of the display unit 115 to display item information based on 
a type of a document input key. 

Then, when the user selects the recognized character data displayed on the 
first display area 71 of the display unit 115 and a SAVE item displayed on the s 
econd display area 75, the controller 101 enables the third display area 73 of the 
20 display unit 1 15 to display the selected character data and SAVE item at step 230 
. Only desired SAVE items associated with the recognized document items can b 
e selected and stored. 



When the "CORRECT" key is inputted, the controller 101 performs step 2 
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40 so that erroneously recognized characters of the recognized character data can 
be corrected. At this time, the correction method displays a group of candidat 
e characters associated with the erroneously recognized characters. If one of th 
e candidate characters is selected, the controller 101 corrects or replaces an erron 
5 eously recognized character with the selected candidate character. However, w 
here an erroneously recognized character cannot be corrected or replaced with an 
y candidate character, the user inputs a handwritten character for correction throu 
gh the input unit 113, and the controller 101 drives the character recognizer 157 s 
o that the handwritten character can be recognized and the error correction proce 
10 ss can be performed. Further, the soft keypad is provided in addition to the han 
dwritten character recognition module. In this case, a method for analyzing soft 
key data inputted from the soft keypad and correcting or replacing erroneously r 
ecognized characters with the soft key data is enabled. 

After the error correction process is completed, the controller 101 stores t 
15 he completely corrected character data in the database 1 59. 

FIG. 3 is a flow chart illustrating the document pick-up process carried ou 
t at the above step 210 shown in FIG. 2, and FIGS. 6A to 6E are views illustratin 
g display screen states of a display unit in a document pickup process. 

Referring to the document pick-up process, the user puts a desired docum 
20 ent to be recognized on an appropriate position, and picks up the desired docume 
nt using the camera 107 provided in the terminal device. An image of the desir 
ed document picked up by the camera 107 is converted into digital data through t 
he image processor 109, and the digital data is displayed on the display unit 115. 
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At this time, if the user of the terminal device inputs a camera adjustment key 
arranged on the key input unit 105 or the input unit 113, the controller 101 senses 
the camera adjustment key input at step 313 and controls the camera 107 at step 
315. The adjustment of the camera 107 can be focus and distance adjustments 
5 . An image of the picked-up document based on the focus and distance adjustm 
ents is displayed on the display unit 1 15 as shown in FIG. 6A. In this case, whe 
n the user inputs a "SHOOT" key of the input unit 113 using the stylus pen, the c 
ontroller 101 senses the key input at step 317 and enables the display unit 1 15 to 
display a still picture corresponding to the document image at a point of "SHOO 
10 T" key input time. Then, the controller 101 pre-processes a document image di 
splayed on the display unit 1 15 at step 319. Here, the pre-processing operation 
removes a noise component contained in the image and corrects an image distorti 
on due to a camera lens and an image skew due to the user's pickup attitude. 

The controller 101 displays the pre-processed document image at step 321 
15 .As the document image displayed at the above step 321 is in a state in which t 
he image has been adjusted and pre-processed at the above steps 313 and 319, it i 
s a clear image after the distortion components (image skew, focus mismatch and 
noise) has been corrected. At this time, the document image displayed on the 
display unit 1 1 5 is shown in FIG. 6C. If the document image displayed on the d 
20 isplay unit 1 15 as shown in FIG. 6C is good, the user inputs the "SAVE" key arr 
anged on the input unit 113 using the stylus pen. If the "SAVE" key is inputted 
, the controller 101 senses the "SAVE" key input at step 323, and store the docu 
ment image along with a name of the document image in the image memory area 
of the memory 103. At this time, while the above steps 323 and 325 are perfor 
25 med, the display unit 115 performs display operations as shown in FIGS. 6C to 6 
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E. However, when the user inputs a "CANCEL" key, the controller 101 senses 
the "CANCEL" key input at step 327 and stops or terminates an operation for dis 
playing the document image. 

In the document pickup process at the above step 210, an image desired b 
5 y the user is inputted through the camera, and various distortions (image skew, fo 
cus mismatch, noise, etc.) are corrected. Furthermore, if the corrected image is 
satisfied, characters are extracted from the picked-up image by the character reco 
gnition process and a determination is made as to whether characters will be stor 
ed as character data (text) or a photo. At this point, if the user of the mobile ter 
10 minal makes a character recognition request, a document recognition process is p 
erformed at step 220. 

FIGS. 4 A and 4B are detailed flow charts illustrating the document recog 
nition process and the SAVE item selection process at the above steps 220 and 2 
30 shown in FIG. 2. FIGS. 7A and 7B show states of the document recognition 
15 process and the SAVE item selection process. 

It is preferable that a document "RECOGNIZE" key inputting for a docu 
ment recognition command is configured according to types of frequently used d 
ocuments in the document recognition process. For example, character informa 
tion recorded on the card can be stored in the phone book of the mobile terminal. 
20 A company name, a company department, a person's name, the person's title, 
an e-mail address, a mobile phone number, etc. are recorded on the card. Thus, 
when information for the phone book is registered in the mobile terminal, it is us 
eful to recognize character information of the card and store the recognized chara 
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cter information in the phone book. Where a character image of a document su 
ch as a card, etc is recognized, it is convenient that a table having an assigned sto 
rage area capable of storing card items or card item information is preset in adva 
nee, the fact that a document to be recognized is a card is sensed by the controlle 
5 r 101 when a card "RECOGNIZE" key is inputted, SAVE items of the card are a 
utomatically displayed, and information item is selectively registered. Thus, in 
the embodiment of the present invention, document "RECOGNIZE" keys based 
on types of documents are provided, table items based on the types of documents 
are assigned to the table in advance, table items associated with a corresponding 
10 document can be displayed when a corresponding document "RECOGNIZE" ke 
y is inputted. Furthermore, where a document not previously preset is recogniz 
ed, a document RECOGNITION key is selected, and document items can be man 
ually set to be processed. In the following embodiment of the present invention 
, we assume that the document is a card. 

15 Referring to FIG. 4A, the controller 101 enables the display unit 1 15 to di 

splay an image of a stored card as shown in FIG. 6E at step 411 before a card "R 
ECOGNIZE" key is inputted at step 413. At this time, if a user inputs the card 
"RECOGNIZE" key of the input unit 1 13, the controller 101 senses the card "RE 
COGNIZE" key input at the above step 413 and drives the character recognizer 1 

20 55 at step 415. At step 417, the character recognizer 155 converts a displayed i 
mage of the card shown in FIG. 6E into character data (text). The controller 10 
1 enables the display unit 1 15 to display the character data as shown in FIG. 7A. 

If the card image has been completely converted into the character data, th 
e controller 101 enables the first display area 71 of the display unit 1 15 to display 
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the character data of the card image, enables the third display area 73 to display 
"ITEM SELECTION" and enables the second display area 75 to display SAVE it 
ems as shown in FIG. 7 A, at step 419. The SAVE items displayed on the secon 
d display area 75 includes a person's name, a company phone number, a mobile 
5 phone number, a home phone number, a facsimile number, an e-mail address, a c 
ompany address, others additional items, etc. When the user selects a character 
data (or sentence) item of the first display area 71 and selects a SAVE item displ 
ayed on the second display area 75 using a stylus pen as shown in FIG. 7B in a di 
splay state shown in FIG. 7 A, the controller 101 senses the character data and SA 
10 VE item selections at step 421, and enables the third display area 73 of the displa 
y unit 1 15 to display the selected SAVE item and character data corresponding to 
the SAVE item as shown in FIG. 7B at step 423. Then, if a "CONFIRM" key is 
inputted from the input unit 113, the controller 101 senses the "CONFIRM" key i 
nput at step 425, and registers the selected SAVE item and the character data cor 
1 5 responding to the SAVE item at step 427. If a "CORRECT" key is inputted fro 
m the input unit 1 13, the controller 101 senses the "CORRECT" key input at step 
429, and performs an error correction process as shown in FIG. 5A at step 43 1 . 
Corrected error data is registered through the above steps 425 and 427. If an 
"END" key is inputted from the input unit 113, the controller 101 senses the "EN 
20 D" key input at step 433 and enables the display unit 1 15 to display all selected S 
AVE items and character data corresponding to the SAVE items at step 435. 

FIG. 5 A is a flow chart illustrating a method for correcting erroneous char 
acter data on a selected item-by-item basis in accordance with the embodiment of 
the present invention. 
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The error correction process preformed at the above step 431 shown in FI 
G. 4A will be described in detail with reference to FIG. 5 A. If the "CORRECT 
" key is inputted, the controller 101 enables the third display area 73 of the displa 
y unit 1 15 to display an erroneously recognized item and character data correspo 
5 nding to the erroneously recognized item as shown in FIG. 8A at step 511. If th 
e user clicks erroneously recognized character data displayed on the first display 
area 71 of the display unit 115 using the stylus pen in a display state as in FIG. 8 
A, the controller 101 senses the erroneously recognized character data click at ste 
p 513, and enables the display unit 1 15 to indicate the character data to be correct 
10 ed as shown in FIG. 8B at step 515. 

In the first embodiment of the present invention, two methods can be used 
to correct erroneously recognized character data. In other words, if an erroneo 
usly recognized character is designated as shown in FIG. 8B, the controller 101 e 
nables the third display area 73 of the display unit 1 15 to display candidate chara 

1 5 cters for correcting the erroneously recognized character, enables the second disp 
lay area 75 to display a recognition window for inputting a handwritten character 
for correcting the erroneously recognized character, and enables the fourth displ 
ay area 77 to display a soft keypad for generating key data for correcting the erro 
neously recognized character. Thus, the user can select a desired character of th 

20 e candidate characters displayed on the third display area 73 or input a desired ch 
aracter in the form of a handwritten character on the second display area 75 to co 
rrect the erroneously recognized character. Furthermore, the soft keypad as wel 
1 as the recognition window for inputting the handwritten character is displayed a 
nd key data generated from the soft keypad is analyzed so that the erroneously re 

25 cognized character can be corrected. 
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If the user selects an arbitrary candidate character of the candidate charact 
ers displayed on the third display area 73 in a sate where the erroneously recogni 
zed character is indicated as shown in FIG. 8B, the controller 101 senses the arbi 
trary character selection at step 517 and corrects or replaces the erroneously reco 
5 gnized character displayed on the first display area 71 with the selected candidate 
character. Further, if the user inputs a handwritten character into the recogniti 
on window of the second display area 75 using the stylus pen in a sate where the 
erroneously recognized character is indicated as shown in FIG. 8B, the controller 
101 senses the handwritten character input at step 521 and drives a handwritten 
10 character recognizer of the character recognizer 155 at step 523. At step 525, th 
e controller 101 corrects or replaces the erroneously recognized character data wi 
th another character data recognized by the character recognizer 155. Furtherm 
ore, if key data is generated from the soft keypad of the fourth display area 77 in 
a sate where the erroneously recognized character is indicated as shown in FIG. 8 
15 B, the controller 101 senses the key data generation at the above step 521, and dr 
ives a soft key recognition module of the character recognizer 123 at the above st 
ep 523. Then, the controller 101 corrects or replaces the erroneously recognize 
d character data with another character data recognized by the character recogniz 
er 155 at the above step 525. 

20 If a "CANCEL" key is inputted, the controller 101 senses the "CANCEL" 

key input at step 527 and cancels, at step 529, the erroneously recognized chara 
cter selected at the above step 513. If an "INSERT" key is inputted, the control 
ler 101 senses the "INSERT" key input at step 531 and determines a position for 
adding (or inserting) the character data at step 533. At this time, the position ca 
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n be a position before or behind the character selected at the above step 513. T 
hen, the controller 101 performs a process of selecting the candidate character or 
inputting the handwritten character and adds (or inserts) a character into the deter 
mined position at step 535. 

5 If the user selects another erroneously recognized character corresponding 

to a selected SAVE item after the candidate character is selected, the erroneousl 
y recognized character is corrected or replaced with the handwritten character, th 
e selected character is cancelled, or another character is added or inserted, the co 
ntroller 101 senses the erroneously recognized character selection at step 527, ret 
10 urns to the above step 515 and repeats the above-described steps. 

If the above-described steps are repeated, the controller 101 corrects chara 
cters corresponding to selected SAVE items. Then, if the "CORRECT END" k 
ey is inputted, the controller 101 senses the "CORRECT END" key input at step 
529, completes the error correction process associated with the selected SAVE it 
15 ems, and returns to the above step 421 shown in FIG. 4A. 

FIGS. 8 A and 8B are shown to explain an operation using candidate chara 
cters and handwritten character recognition in the character data correction proce 
ss. However, the erroneously recognized character can be corrected using only 
a handwritten character without using the candidate characters. FIG. 8D is a flo 
20 w chart illustrating a method for correcting character data by inputting handwritt 
en characters or soft keys without using the candidate characters. 



In the method shown in FIGS. 4A and 5 A, a SAVE item is selected, the se 
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lected SAVE item and character data corresponding to the selected SAVE item a 
re registered if the character data does not have any errors. The character data i 
s corrected if the character data has errors, and the selected SAVE item and the c 
orrected character data are registered. 

5 Character data displayed on the first display area 71 is selected with the st 

ylus pen as shown in FIG. 7B, and a SAVE item, corresponding to the character 
data, displayed on the second display area 75 is selected with the stylus pen. Th 
en, the selected SAVE item and the character data corresponding to the selected 
SAVE item are displayed on the third display area 73. At this time, the "CONF 

10 IRM" key is clicked with the stylus pen as shown in FIG. 7B if the SAVE item a 
nd the character data displayed on the third display area 73 are appropriate. In t 
his case, the SAVE item and the character data displayed on the third display are 
a 73 are registered. However, if the character data displayed on the third displa 
y area 73 has an error, the "CORRECT 55 key is clicked with the stylus pen as sho 

15 wn in FIG. 8 A. Then, if an erroneous character displayed on the first display ar 
ea 71 as shown in FIG. 8B is clicked with the stylus pen, the clicked character is 
enlarged and displayed, and candidate characters associated with the erroneously 
recognized character are displayed on the third display area 73. Then, the recog 
nition window for inputting the handwritten character is displayed on the second 

20 display area 75, and the soft keypad is displayed on the fourth display area 77. I 
n this state, a method for correcting the erroneously recognized character selects 
one of the candidate characters displayed on the third display area 73, inputs a ha 
ndwritten correction character into the recognition window of the second display 
area 75, or inputs character key data for the error correction process through the s 

25 oft keypad of the fourth display area 77. Further, where the character is cancell 
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ed or inserted, the "CANCEL" key is inputted or the "INSERT" key is inputted. 

If another erroneously recognized character associated with the selected SAVE 
item exists, the above-described procedure is repeated. If the correction process 

has been completed, the user clicks the "CORRECT END" key with the stylus 
5 pen. If so, the method returns to the display state shown in FIG. 7A so that the 
next SAVE item can be selected. 

The method for correcting an erroneously recognized character through th 
e handwritten character input, the candidate character selection and the soft keyp 
ad has been described, but the method can be implemented through only the hand 
1 0 written character input or the soft keypad. Furthermore, the method can be impl 
emented using the candidate character selection and the handwritten character re 
cognition or can be implemented using the candidate character selection and the 
soft keypad. 

FIGS. 4B and 5B are flow charts illustrating a method for carrying out oth 
15 er SAVE item selection and error correction processes in accordance with the fir 
st embodiment of the present invention. 

The method for carrying out other SAVE item selection and error correcti 
on processes will be described with reference to FIG. 4B. The character recogn 
ition and SAVE item selection processes shown in FIG. 4B are almost identical t 
20 o those shown in FIG. 4A, except that a signal indicating that an error associated 
with a corresponding SAVE item exists is displayed in FIG. 4B before the error c 
orrection process if the "CORRECT" key is inputted. In other words, if the "C 
ORRECT" key is inputted in the item selection process, the controller 101 senses 
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the "CORRECT" key input at step 429, and enables the display unit 1 15 to displ 
ay the signal indicating that an error of character data associated with a correspo 
nding SAVE item exists at step 450. Then, the method returns to step 421. Th 
e method shown in FIG. 4B is almost identical to that shown in FIG. 4 A except f 
5 or the correction process. Thus, if a character recognition error associated with 
the selected SAVE item in the SAVE item selection process subsequent to the ch 
aracter recognition process exists when the character recognition and SAVE item 

selection processes are carried out using the method shown in FIG. 4B, the sign 
al indicating that an error associated with a corresponding SAVE item exists is di 
10 splayed and the method returns to another step. Otherwise, if no character reco 
gnition error exists, a corresponding SAVE item and characters associated with t 
he SAVE item are registered. 

The method can be implemented by the SAVE item selection process subs 
equent to the character recognition process without performing character confirm 
15 ation and correction operations on a SAVE item-by-item basis. In other words, 
after all desired items are selected from the recognized document without the cha 
racter confirmation and correction operations, a method for confirming character 
data of selected SAVE items and collectively correcting all errors associated wit 
h the character data can be implemented. 

20 FIG. 5B is a flow chart illustrating a method for correcting erroneous char 

acter data associated with SAVE items after the character recognition and SAVE 
item selection processes as in the method shown in FIG. 4B. 

The error correction method will be described with reference to FIG. 5B. I 
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f the "CORRECT" key is inputted, the controller 101 senses the "CORRECT" ke 
y input at step 551, enables the second display area 75 of the display unit 1 15 to 
display erroneously recognized items and enables the first display area 71 of the 
display unit 1 1 5 to display character data corresponding to the erroneously recog 
5 nized items at step 553. If the user clicks erroneously recognized character data 
displayed on the first display area 71 of the display unit 115 using the stylus pen 
in a display state as in FIG. 8 A, the controller 101 senses the erroneously recog 
nized character data click at step 513, and enables the display unit 1 15 to indicate 
the erroneously recognized character data as shown in FIG. 8B at step 515. Th 
10 en, if the user clicks a SAVE item for correcting the erroneously recognized char 
acter data using the stylus pen, the controller 101 senses the SAVE item click at s 
tep 555, proceeds to step 557 and carries out the process shown in FIG. 5A. 

The process shown in FIG. 5A corrects erroneously recognized character 
data associated with character data of the selected SAVE item. If the correction 

15 process is completed, the controller 101 stores a corresponding SAVE item and 
corrected character data at step 559. If the user selects the next SAVE item afte 
r the correction process for the character data associated with the selected SAVE 
item is completed, the controller 101 senses the next SAVE item selection at step 
561, returns to the above step 557 and repeats the operation of correcting the err 

20 oneously recognized character data associated with the selected SAVE item. T 
he SAVE items associated with the erroneously recognized character data are seq 
uentially selected so that the character data correction operation can be performe 
d. Then, if the character data correction process for all SAVE items is complete 
d, the user inputs the "CORRECT END" key of the input unit 113. The control 

25 ler 101 senses the "CORRECT END" key input at step 561 and enables the displ 
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ay unit 1 1 5 to display SAVE items and corrected character data corresponding to 
the SAVE items and saves them at step 563. 

If the character recognition, SAVE item selection and error correction pro 
cesses are completed, all desired information items recorded on the card can be i 
5 nputted. In this case, character data of the selected SAVE items is displayed as 
shown in FIG. 9A. In this state, if the user clicks the "END" key using the stylu 
s pen, the controller 101 senses the fact that the card recognition is completed an 
d enables the display unit to display all SAVE items and the character data corres 
ponding to the SAVE items on a single screen as shown in FIG. 9B. Then, the 

10 character data displayed as shown in FIG. 9B is stored in the database 131. In o 
ther words, the database 1 59 saves SAVE item-based data recognized from the d 
ocument in a desired storage area thereof if the input, recognition and correction 
processes are completed. The database 159 can include various storage spaces 
capable of storing a phone book, memos, other applications, etc. If all desired d 

15 ata items are stored, a program is terminated. 

In accordance with the second embodiment of the present invention, an er 
ror is corrected on a SAVE item-by-item basis, and a speech recognition method 
is used in the document recognition and error correction processes. 

FIG. 10 is a flow chart illustrating an operation in accordance with the sec 
20 ond embodiment of the present invention. 

Referring to FIG. 10, the controller 101 enables the camera 107 to pick up 
a document image at step 210. At this time, the image picked up by the earner 
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a 107 is processed by the image processor 109, the processed image is converted 
into digital data, and the digital data is displayed on the display 115. If a still-pi 
cture capture command is issued in a state where the picked-up image is displaye 
d on the display unit 115, the controller 101 enables the display unit 1 15 to displa 
5 y a still picture, and saves the image displayed on the display unit 115 in an imag 
e memory area of the memory 103. At this time, the image displayed on the dis 
play unit 115 can be a moving picture, and can be character image data such as a 
card, etc. 

The user of the mobile terminal inputs a document "RECOGNIZE" key c 
orresponding to a currently displayed document through the input unit 113. If s 
o, the controller 101 drives the character recognizer 155 at step 220. The docu 
ment recognizer 155 recognizes the character images from the image displayed o 
n the display unit 115 and converts the recognized character images into characte 
r data. The controller 101 enables the first display area 71 of the display unit 1 1 
5 to display the character data recognized by the character recognizer 155, and en 
ables the second display area 75 of the display unit 1 15 to display SAVE items b 
ased on types of document input keys. 

Then, if the user selects the recognized character data displayed on the firs 
t display area 71 of the display unit 115 and selects a SAVE item displayed on th 
20 e second display area 75, the controller 101 enables the third display area 73 of t 
he display unit 1 1 5 to display the selected character data and SAVE item at step 
230. At this time, the SAVE item selection process uses a method for selecting 
a displayed SAVE item through the input unit 113 and a method for selecting a S 
AVE item through the speech recognizer 153. 
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After the SAVE item is selected, an operation of correcting character data 
associated with a corresponding SAVE item is carried out if the character data of 
the selected SAVE item has an error. At this time, an error correction request 
can be made by selecting at least one erroneous character using the input unit 1 1 
5 3 or by speech using the speech recognizer 153. If the error correction request i 
s made through the above-described method, the controller 101 senses the error c 
orrection request at step 241 and corrects the erroneous character of the recogniz 
ed character data at step 240. 

If the correction process is completed, the controller 101 stores the correct 
10 ed character data as character data associated with a corresponding SAVE item i 
n the database 159. 

FIG. 1 1 shows the document pickup process performed at the above step 
210 shown in FIG. 10; and FIGS. 6A to 6E are views of the images picked up by 
the document pickup process and displayed by the display unit 115. 

1 5 The document pickup process will now be described. The user places a 

document to be recognized at an appropriate position and picks up an image of th 
e document using the camera 107 of the mobile terminal. If so, the controller 1 
01 enables the display unit 1 15 to display a preview image as shown in FIGS. 6A 
and 6B at step 651. At this time, if the user of the terminal device inputs a ca 
20 mera adjustment key arranged on the key input unit 105 or the input unit 113, the 
controller 101 senses the camera adjustment key input at step 653 and controls t 
he camera 107 at step 315. The adjustment of the camera 107 can be focus an 
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d distance adjustments. An image of the picked-up document based on the focu 
s and distance adjustments is displayed on the display unit 1 15 as shown in FIG. 
6A. In this case, when the user inputs a "SHOOT" key of the input unit 113 usi 
ng the stylus pen, the controller 101 senses the key input at step 655 and enables 
5 the display unit 1 1 5 to display a still picture corresponding to the document imag 
e at a point of "SHOOT" key input time. Then, the controller 101 pre-processes 
a document image displayed on the display unit 1 15 at step 657. Here, the pre 
-processing operation removes a noise component contained in the image and cor 
rects an image distortion due to a camera lens and an image skew due to the user' 
10 s pickup attitude. 

The controller 101 displays the pre-processed document image at step 659 
. As the document image displayed at the above step 659 is in a state in which t 
he image has been adjusted and pre-processed at the above step 653, it is a clear i 
mage after the distortion components (image skew, focus mismatch and noise) ha 

15 s been corrected. At this time, the document image displayed on the display uni 
t 1 1 5 is shown in FIG. 6C. If the document image displayed on the display unit 
1 15 as shown in FIG. 6C is good, the user inputs the "SAVE" key arranged on th 
e input unit 113 using the stylus pen. If the "SAVE" key is inputted, the control 
ler 101 senses the "SAVE" key input at step 661, and store the document image a 

20 long with a name of the document image in the image memory area of the memo 
ry 103. At this time, the document image displayed on the display unit 115 is th 
e same as that shown in FIG. 6E. 

Then, if the user clicks a card "RECOGNIZE" key, the controller 101 sen 
ses the card "RECOGNIZE" key input at step 663 and performs the document re 
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cognition process performed at the above step 220. Otherwise, a currently displ 
ayed document image is stored at step 665 and the process is terminated. 

In the document pickup process at the above step 210, an image desired b 
y the user is inputted through the camera, and various distortions (image skew, fo 
5 cus mismatch, noise, etc.) are corrected. Furthermore, if the corrected image is 
satisfied, characters are extracted from the picked-up image by the character reco 
gnition process and a determination is made as to whether characters will be stor 
ed as character data (text) or a photo. At this point, if the user of the mobile ter 
minal makes a character recognition request, a document recognition process is p 
10 erformed at step 220. 

FIG. 12 is a detailed flow chart illustrating the document recognition proc 
ess and the SAVE item selection process at the above steps 210 and 230 shown i 
n FIG. 10. FIGS. 7A and 7B show states of the document recognition process a 
nd the SAVE item selection process. 

15 Referring to FIG. 12, the controller 101 enables the display unit 1 15 to dis 

play an image of a stored card as shown in FIG. 6E before a card "RECOGNIZE 
" key is inputted. At this time, if the user inputs the card "RECOGNIZE" key o 
f the input unit 113, the controller 101 senses the card "RECOGNIZE" key input 
and drives the character recognizer 155 at step 751. At step 753, the character r 

20 ecognizer 155 converts the displayed card image shown in FIG. 6E into character 
data or text, and the controller 101 enables the display unit 1 15 to display the ch 
aracter data or text as shown in FIG. 7A. If the card image is converted into ch 
aracter data, the controller 101 enables the first display area 71 of the display uni 
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t 1 15 to display the character data of the card image, enables the third display are 
a 73 to display a selected SAVE item and character data corresponding to the sel 
ected SAVE item, and enables the second display area 75 to display SAVE items 
as shown in FIG. 7 A. When the user selects a character data (or sentence) ite 
5 m of the first display area 71 and selects a SAVE item of the second display area 
75 using a stylus pen as shown in FIG. 7B in a state where the recognized charac 
ter data is displayed as shown in FIG. 7A, the controller 101 senses the character 
data and SAVE item selections at step 757, and enables the third display area 73 
of the display unit 1 15 to display the selected SAVE item and character data corr 
10 esponding to the SAVE item as shown in FIG. 7B. At the above step 757, the S 
AVE item selection process can be executed by speech. In this case, the user of 
the mobile terminal can select a speech recognition mode through the input unit 
1 13 or the key input unit 105 and input a desired SAVE item by speech. 

If the "CORRECT" key is inputted from the input unit 1 13 in the state wh 
15 ere the character data is displayed, the controller 101 senses the "CORRECT 55 ke 
y input at step 759, and performs the error correction process at step 761 . Other 
wise, if the "CORRECT 55 key is not inputted, the controller 101 determines whet 
her a key for selecting the next SAVE item is inputted. If the key for selecting t 
he next SAVE item is inputted, the controller 101 senses the key input at step 25 
20 1, and selects the next SAVE item at the above step 755. However, if an "END 
55 key input is sensed at the above step 251, the controller 101 saves character dat 
a corresponding to the selected SAVE items in the database 159 at step 253, and 
terminates the document recognition process. 



Another document recognition process in accordance with the second emb 
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odiment of the present invention performs an operation of driving the character r 
ecognizer to convert a character image contained in the input document image int 
o text. After the text is displayed on the display unit 115, the controller 101 alio 
ws the user to select desired characters. At this time, a storage area for storing t 
5 he selected characters (corresponding to a name, an address, a company, etc.) is d 
esignated, and copied characters are stored in the assigned storage area. A proc 
ess for designating the storage area during the SAVE item selection process will 
be described in detail in a SAVE item selection process shown in FIG. 13. If a 
character to be corrected is contained in the recognized characters, the error corre 
10 ction process is carried out. Otherwise, a determination is made as to whether a 
n additional SAVE item to be stored exists. If an additional SAVE item is to be 
stored, the SAVE item selection process is repeated, character data is stored in t 
he database in the storage process, and a program is terminated. 

FIG. 13 is a detailed flow chart illustrating the SAVE item process perfor 
15 med at the above steps 755 and 757 shown in FIG. 12. 

Referring to FIG. 13, a character recognition process is carried out and rec 
ognized character data is displayed on the display unit 1 15 as shown in FIG. 7A. 

At this time, the user can select SAVE items displayed on the second display a 
rea 75 using the stylus pen or select a speech recognition mode through the input 
20 unit 1 13 or key input unit 105. If the speech recognition mode is selected, the c 
ontroller 101 senses the speech recognition mode selection at step 771 and allow 
s the user to input a desired SAVE item and data by speech through a recording b 
utton at step 773. If so, the controller 101 applies a speech signal received thro 
ugh the audio processor 1 1 1 to the speech recognizer 153 at the above steps 773 
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and 775, and drives the speech recognizer 155 so that the received speech signal 
can be recognized. Then, the controller 101 enables the display unit 1 15 to disp 
lay character data associated with a SAVE item corresponding to a speech recog 
nition signal as shown in FIG. 7B and saves the character data at step 777. 

5 Upon sensing the SAVE item selection using the stylus pen at step 77 1 , th 

e controller 101 enables the display unit 1 15 to display SAVE items as shown in 
FIG. 7A at step 779. If a desired SAVE item is selected with the stylus pen, a s 
elected SAVE item and character data (text) are displayed as shown in FIG. 7B a 
t step 781, and the character data (text) is stored in a storage area corresponding t 
10 o the selected SAVE item at step 783. 

As described above, the SAVE item selection method is divided into the S 
AVE item selection method using the speech recognition and the SAVE item sel 
ection method using the stylus pen. After the recording button in the SAVE ite 
m selection method using the speech recognition is pressed, the user pronounces 

15 a desired SAVE item of the SAVE items displayed as shown in FIG. 7A to select 
the desired SAVE item through the speech recognizer 153. If an "ITEM ADD 
ITION" is selected, a desired additional SAVE item is received from the user an 
d the additional SAVE item is added to a SAVE item table before the next proces 
s is carried out. Furthermore, in the SAVE item selection method using the styl 

20 us pen, a desired SAVE item of the SAVE items displayed on the display unit 1 1 
5 as shown in FIG. 7 A is clicked and selected. The above-described two metho 
ds are not sequentially carried out, but one of the two methods can be selectively 
carried out according to the user's selection. 
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FIG. 13 explains an example of selecting a desired SAVE item and charac 
ter data using the speech recognition or stylus pen. The desired SAVE item can 
be selected by the speech recognition and the character data can be selected wit 
h the stylus pen. Furthermore, the desired SAVE item can be selected with the 
5 stylus pen and the character data can be selected by the speech recognition. 

FIGS. 14A to 14D are detailed flow charts illustrating the process of corre 
cting erroneous character data on a selected SAVE item-by-item basis performed 
at the above step 240 shown in FIG. 10. 

Referring to FIG. 14 A, the controller 101 enables the third display area 73 
10 of the display unit 1 1 5 to display the selected SAVE item and character data cor 
responding to the selected SAVE item as shown in FIG. 8A when a desired SAV 
E item is selected. When the character data corresponding to the selected SAV 
E item is erroneously recognized, the user clicks the "CORRECT" key using the 
stylus pen or selects the speech recognition mode to make a correction request. 
15 The controller 101 senses the correction request, receives candidate characters cl 
osest to a character recognized by the character recognizer 155, and enables the t 
hird display area 73 of the display unit 1 15 to display the candidate characters. 
At this time, the controller 101 enables the third display area 73 of the display un 
it 1 15 to display the candidate characters for correcting the erroneously recognize 
20 d character, enables the second display area 75 of the display unit 1 1 5 to display 
a recognition window for inputting handwritten characters to correct the erroneo 
usly recognized character or enables the fourth display area 77 of the display unit 
1 1 5 to display a soft keypad as shown in FIG. 8B. If a desired character is con 
tained in the candidate characters displayed on the third display area 73 of the dis 
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play unit 115, the user of the mobile terminal clicks the desired candidate charact 
er using the stylus pen. Thus, if an arbitrary candidate character is selected fro 
m the candidate characters, the controller 101 senses the candidate character sele 
ction at step 815, and corrects or replaces the erroneously recognized character di 
5 splayed on the first display area 71 with the selected candidate character at step 8 
17. 

If the desired character is not contained in the candidate characters display 
ed on the third display area 73, the user of the mobile terminal can select the spee 
ch recognition mode, use the handwritten character recognition window displaye 

10 d on the second display area 75, or use the soft keypad displayed on the fourth di 
splay area 77. At this time, if the user selects the speech recognition mode thro 
ugh the input unit 1 13 or the key input unit 105, the controller 101 performs the 
operation associated with FIG. 14B at step 820. If the user inputs a handwritten 
character into the handwritten character recognition window displayed on the se 

15 cond display area 75, the controller 101 performs the operation associated with F 
IG. 14C at step 850. 

The correction process refers to a value of data extracted by the character 
recognizer 123. When one character is recognized by the character recognition 
process as in FIG. 13, the character recognizer 155 decides a candidate character 
20 closest to the input character as character data associated with a corresponding S 
AVE item, and keeps other candidate characters close to the input character. In 
the error correction process shown in FIG. 14A, the user requests that the chara 
cter recognizer 155 provide candidate characters associated with a character to b 
e corrected, and hence the third display area 73 of the display unit 115 displays th 
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e candidate characters provided by the character recognizer 155. At this time, if 
a desired candidate character is contained in the candidate characters, the user o 
f the mobile terminal selects the desired character using the stylus pen and correc 
ts or replaces the erroneously recognized character with the selected candidate ch 
5 aracter. Otherwise, if no desired character is in the candidate characters, the us 
er of the mobile terminal executes the speech recognition process shown in FIG. 
14B, the handwritten character recognition process shown in FIG. 14C or the sof 
t key recognition process shown in FIG. 14D. These processes are executed on 
one screen. While the handwritten character recognition window and the soft k 

10 eypad are displayed on a lower part of the display unit 115 provided in the mobil 
e terminal, the display unit 115 waits for the user to select the handwritten charac 
ter recognition window or the soft keypad. Where the user presses the recordin 
g button, the speech recognizer 153 is driven. Accordingly, the character recog 
nizer 155 is designed so that printed characters, handwritten characters and soft k 

15 eys can be recognized. 

Referring to FIG. 14B, the speech recognizer 153 must be differently driv 
en according to language. The speech recognizer 153 receives a speech signal 
on a letter-by-letter or character-by-character basis rather than on a word-by-wor 
d basis. A word consists of at least one letter in English, and an English letter c 

20 orresponds to a character. However, a character consists of several character el 
ements or letters in Korean. For example, the word "KOREA" in English consi 
sts of five letters, while the word "&J"^" (Hankook)" in Korean consists of two c 
haracters that consist of three letters, respectively. In relation to Korean langua 
ge, the speech recognizer 153 must receive speech signals corresponding to a des 

25 ired character on a letter-by-letter basis if the speech recognizer 153 is not an unr 
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estricted speech recognition engine. Thus, a language mode is first selected in t 
he speech recognition mode, and a determination is made as to whether the speec 
h recognizer 153 is the unrestricted speech recognition engine if the language mo 
de is based on Korean language. 

5 In the speech recognition process, the controller 101 determines whether a 

letter to be corrected is English or Korean at step 821. If the letter to be correc 
ted is English, the user of the mobile terminal selects an English mode, presses th 
e recording button, and inputs character data for correction by speech. If so, the 
controller 101 senses the speech input corresponding to the English character da 
10 ta at step 835 and drives the speech recognizer 153 at step 837. The speech rec 
ognizer 153 recognizes the English character data of the speech outputted from t 
he audio processor 111 and outputs the recognized English character data to the c 
ontroller 101. Then, the controller 101 corrects the character data correspondin 
g to the selected SAVE item with the recognized English character data and deter 
15 mines whether the next character data must be corrected at step 839. At this tim 
e, if character data to be corrected exists, the controller 101 returns to the above s 
tep 835 and repeats the above-described procedure. If no character data to be c 
orrected exists, the controller 101 performs the above step 251 shown in FIG. 10. 

On the other hand, if a character to be corrected is Korean, the controller 
20 101 determines whether the speech recognizer 155 is an unrestricted speech reco 
gnition engine. If the speech recognizer is the unrestricted speech recognition e 
ngine, the controller 101 performs the above step 823, and performs the speech r 
ecognition operation of the Korean character while performing the above steps 8 
35 to 839. The Korean language speech recognition is performed on a characte 
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r-by-character basis. 

On the other hand, if the speech recognizer 153 is not an unrestricted spee 
ch recognition engine, the controller 101 enables the speech recognizer 153 to pe 
rform a Korean language speech recognition process on a letter-by-letter basis at 
5 step 825. In this case, the user presses the recording button where the character 
data corresponding to the selected SAVE item is corrected, sequentially inputs sp 
eech signals corresponding to letters configuring Korean character data, and pres 
ses the completion button if the speech signal input for the letters corresponding t 
o a character is completed. If the speech signals corresponding to letters config 

10 uring the character are inputted, the controller 101 receives the input speech sign 
als at the above step 825, and drives the speech recognizer 129 so that the receiv 
ed speech signals can be recognized. After the speech signals corresponding to 
the letters for character correction are recognized, the controller 101 senses the c 
ompletion of the speech signal input at step 829, combines the letters at the abov 

15 e step 829 to form a character at step 83 1 , and corrects or replaces an erroneous c 
haracter with the character formed by the combined letters. Then, the controller 

101 determines whether the next character data must be corrected at step 833. 
If another character to be corrected exists, the controller 101 returns to the above 
step 825 so that the above-described steps can be repeated. Otherwise, if no ch 

20 aracter to be corrected exists, the above step 251 shown in FIG. 10 is carried out. 

Where the error correction process cannot be carried out using the stylus p 
en, the error correction process can be carried out using the speech recognition. I 
n the speech recognition process, a determination is made as to whether a charact 
er to be corrected consists of at least two letters. For example, a Korean charact 
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er consists of a plurality of letters including at least one consonant and at least on 
e vowel, i.e., a set of initial and medial letters or a set of initial, medial and final 1 
etters. Since an unrestricted speech recognizer requiring very large capacity sof 
tware cannot be embedded in the mobile terminal (e.g., a PDA), the Korean reco 
5 gnition for other characters rather than previously inputted characters is disabled. 
Thus, a correctable range in Korean is formed on a letter-by-letter basis (i.e., o 
n an initial/media/final letter basis). The character correction process is achieve 
d on a letter-by-letter basis. If the unrestricted speech recognizer can be imple 
mented within the mobile terminal in the future, the character correction process 

10 can be carried out on a letter-by-letter or character-by-character basis. Meanwh 
ile, since English letters or special characters can be corrected on a letter-by-lette 
r or character-by-character basis, the user selects a desired alphabet or special ch 
aracter, presses the recording button, and pronounces a desired character through 
the speech recognizer so that an erroneous character can be corrected or replace 

15 d with the pronounced character. If the correction process is completed, the me 
thod returns to the SAVE item selection process shown in FIG. 10. 

An error correction process using a handwritten character recognition pro 
cess will be described with reference to FIG. 14C. The controller 101 enables t 
he display unit 1 1 5 to indicate an erroneously recognized character as shown in F 
20 IG. 8B at step 851 . If a handwritten character is inputted into the recognition w 
indow of the second display area 75 with the stylus pen, the controller 101 senses 
the handwritten character input at step 853, and drives the character recognizer 
155 so that the inputted handwritten character can be recognized at step 855. T 
he controller 101 corrects or replaces erroneously recognized character data corre 
25 sponding to the selected SAVE item with character data recognized by the charac 
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ter recognizer 155. The controller 101 determines whether the next character d 
ata must be corrected at step 857. If another character to be corrected exists, th 
e controller 101 returns to the above step 853 so that the above-described steps c 
an be repeated. Otherwise, if no character to be corrected exists, the above step 
5 251 shown in FIG. 10 is carried out. 

The error correction process using the handwritten character recognition i 
s carried out through the handwritten character recognition window loaded on th 
e second display area 75 of the display unit 115. Where an erroneous character 
cannot be corrected by the error correction process shown in FIG. 14A, the user c 
1 0 an perform the correction operation by directly inputting a desired character into 
the handwritten character recognition window. 

The error correction process by the soft key recognition will be described 
with reference to FIG. 14D. The controller 101 enables the display unit 1 15 to i 
ndicate an erroneously recognized character as shown in FIG. 8D and to display t 

15 he soft keypad on the fourth display area 77. At this time, if key data is inputte 
d or received from the soft keypad, the controller 101 senses the key data recepti 
on at step 873 and drives the soft key recognizer of the character recognizer 155 
at step 875 so that characters corresponding to the input key data can be recogniz 
ed. If so, the controller 101 corrects or replaces erroneously recognized charact 

20 er data of the selected SAVE item with character data recognized by the soft key 
recognizer of the character recognizer 155. The controller 101 determines whet 
her the next character data must be corrected at step 877. If another character t 
o be corrected exists, the controller 101 returns to the above step 853 so that the 
above-described steps can be repeated. Otherwise, if no character to be correct 
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ed exists, the above step 251 shown in FIG. 10 is carried out. 

The error correction by the soft key recognition is carried out through the 
soft keypad loaded on the fourth display area 77 of the display unit 115. Where 
an erroneous character cannot be corrected by the error correction process show 
5 n in FIG. 14A, the user can perform the correction operation by directly inputting 
a desired character through soft keys of the soft keypad. 

If the SAVE item selection process and the error correction process for ch 
aracter data corresponding to a selected SAVE item are completed, the user of th 
e mobile terminal inputs the "END" key through the input unit 113. Then, the c 
10 ontroller 101 senses the "END" key input at step 251 and saves a result of the do 
cument recognition in the database 159. The database 159 registers selected SA 
VE items and character data corresponding to the selected SAVE items in an add 
ress designated by the user. 

The database 159 saves SAVE item-based data recognized from the docu 
1 5 ment in a desired storage area thereof if the input, recognition and correction pro 
cesses are completed. The database 159 can include various storage spaces cap 
able of storing a phone book, memos, other applications, etc. If all desired data 
items are completely stored, a program is terminated. 

In accordance with the second embodiment of the present invention, SAV 
20 E items associated with a recognized document after document recognition are se 
lected, an erroneous character is corrected if character data of a selected SAVE it 
em has an error, and the next SAVE item is selected. Thus, an erroneous charac 
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ter is corrected and the corrected character is stored while character data for the r 
ecognized document is stored on a SAVE item-by-item basis. In the second em 
bodiment of the present invention, a speech recognizer can be used when a SAV 
E item is selected or an erroneous character is corrected. 

5 In the error correction process in accordance with the second embodiment 

of the present invention, a candidate character is first selected, and an erroneous 
character is corrected by the selected candidate character. Where the error cor 
rection process using the candidate character is disabled, it has been explained th 
at the error correction can be achieved through the speech recognition or handwri 

10 tten character, and the soft key recognition. In the erroneous character correctio 
n process, some methods of the candidate character selection method, the speech 
input method using the speech recognition, the handwritten character input meth 
od and the character input method using the soft keypad can be selectively imple 
mented. In other words, the erroneous character correction method allows the u 

15 ser to directly input a speech signal, a handwritten character and a soft key witho 
ut selecting a candidate character. Furthermore, the erroneous character correct 
ion method using the candidate character selection, the speech recognition and th 
e handwritten character recognition has been described, but it can be implemente 
d using only the candidate character selection and speech recognition method, th 

20 e speech recognition and handwritten character recognition method, or the speec 
h recognition and soft key recognition method. 

In the embodiments of the present invention, it is assumed that the above- 
described document is a card bearing a person's name and other information, but 
the embodiments of the present invention can be applicable to other documents r 
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ather than the card. 

[Effects of the Invention] 

Where information of the document is registered in a device such as a mo 
bile terminal, an image of the document is scanned, and character data can be reg 
istered through character recognition and/or speech recognition, such that a mani 
pulation of an input unit through the mobile terminal can be simplified and an err 
oneous character can be conveniently corrected in the character recognition or sp 
eech recognition. Since the information of the document can be inputted throug 
h the character and speech recognition methods, a large capacity of the document 
information can be efficiently inputted. 
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[Claims] 

1 . A device for storing document information using a camera in a mobile t 
erminal, comprising: 

the camera for picking up document information; 
5 a character recognizer for converting a document image picked up by the 

camera into text; 

an input unit for input signals for selecting desired items to be stored from 
recognized text characters, designating erroneously recognized text characters f 
rom the selected items, and generating text characters for correcting the designat 
10 ed text characters; 

an error corrector for correcting the text characters of corresponding items 
when error correction input signals are generated; 

a database for storing the selected items and character data; 
a controller for driving the character recognizer when a document image i 
15 s generated from the camera, displaying the selected items and corresponding tex 
t characters when the item selection input signals are generated, driving the error 
corrector when the error correction input signals are generated, replacing the erro 
neously recognized text characters with the corrected text characters, and storing 
the selected items and text characters in the database when a character input is co 
20 mpleted; and 

a display unit for displaying the document information while the documen 
t is stored. 



2. The device as set forth in claim 1, further comprising: 

a speech recognizer for generating the input signals for selecting the items 
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, designating the erroneously recognized text characters, and generating the text c 
haracters for correcting, the speech recognizer converting a received speech sign 
al into a text character. 

3. The device as set forth in claim I, wherein the character recognizer furt 
5 her comprises a handwritten character recognizer, the handwritten character reco 

gnizer recognizing a handwritten character input associated with the erroneously 
recognized text characters when an error is corrected. 

4. The device as set forth in claim 1, wherein the camera can carry out foe 
us and distance adjustments. 

10 5. A device for storing document information using a camera in a mobile t 

erminal, comprising: 

the camera for picking up document information; 

a display unit having a first display area for displaying text characters, a s 
econd display area for displaying items, and a third display area for displaying a s 
1 5 elected item and text characters of the selected item, the display unit displaying a 
menu for character recognition and item selection as a touch screen module, an 
d displaying a menu for correcting the text characters of the selected item; 

a character recognizer driven when the character recognition menu is sele 
cted for converting a document image picked up by the camera into text; 
20 an error corrector for correcting the characters of the item displayed on th 

e third display area when the correction menu is selected; and 

a database for storing selected items and character data corresponding to t 
he selected items when the correction is completed. 
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6. The device as set forth in claim 5, wherein the character recognizer furt 
her comprises a handwritten character recognizer, and wherein the display unit di 
splays a handwritten character window on the third display area, and the characte 
r recognizer recognizes a handwritten character inputted into the handwritten cha 
racter window and generates the corrected text character. 

7. The device as set forth in claim 5, further comprising: 

a speech recognizer for receiving a speech signal for correcting a design 
ated text character in a correction mode and correcting the designated text charac 
ter on the basis of the speech signal. 

8. A method for storing document information using a camera in a mobile 
terminal, comprising the steps of: 

displaying a document image picked up by the camera; 

carrying out a character recognition operation for the displayed document 
image in a document recognition mode to convert the displayed document image 
into character data, displaying the character data on a first display area, and displ 
aying document items on a second display area; 

selecting an item to be stored from the displayed items and selecting and d 
isplaying character data of the selected item; and 

repeating the above steps and storing selected items and character data uni 
ts corresponding to the selected items. 

9. The method as set forth in claim 8, further comprising the step of: 
correcting erroneously recognized character data after selecting the item a 
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nd character data, the step of correcting the erroneously recognized character dat 
a comprising the steps of: 

when an error correction request is made, displaying candidate characters 
of an erroneously recognized character; and 

replacing or correcting the erroneously recognized character with one sele 
cted from the displayed candidate characters. 

10. The method as set forth in claim 8, further comprising the step of: 
correcting erroneously recognized character data after selecting the item a 

nd character data, the step of correcting the erroneously recognized character dat 
a comprising the steps of: 

when an error correction request is made, displaying a handwritten charac 
ter recognition window on the second display area; 

when a character is inputted into the handwritten character recognition wi 
ndow, recognizing the inputted character; and 

replacing or correcting the erroneously recognized character with the reco 
gnized character. 

1 1 . The method as set forth in claim 8, further comprising the step of: 
correcting erroneously recognized character data after selecting the item a 

nd character data, the step of correcting the erroneously recognized character dat 
a comprising the steps of: 

when an error correction request is made, displaying candidate characters 
of the erroneously recognized character on a third display area; 

replacing or correcting the erroneously recognized character data with one 
selected from the candidate characters; 
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when a desired character is not present in the candidate characters, display 
ing a handwritten character recognition window on the second display area; 

when a handwritten character is inputted into the handwritten character re 
cognition window, recognizing the inputted handwritten character; and 

replacing or correcting the erroneously recognized character with the reco 
gnized character. 

12. The method as set forth in claim 8, further comprising the step of: 
correcting erroneously recognized character data after selecting the item a 

nd character data, the step of correcting the erroneously recognized character dat 
a comprising the steps of: 

when an error correction request is made, driving a speech recognizer; 

allowing the speech recognizer to recognize an inputted speech signal and 
to convert the recognized speech signal into character data; and 

replacing or correcting the erroneously recognized character data with the 
character data based on the recognized speech signal. 

13. A method for storing document information using a camera in a mobil 
e terminal, comprising the steps of: 

displaying a document image picked up by the camera; 

carrying out a character recognition operation for the displayed document 
image when a document "RECOGNIZE" key is received, converting the docume 
nt image into character data, displaying the character data on a first display area, 
and displaying document items on a second display area; 

selecting an item to be stored from the displayed items, selecting character 
data of the selected item, and displaying the selected item and the selected chara 
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cter data on a third display area; 

displaying candidate characters of an erroneously recognized character wh 
en a "CORRECT 55 key is inputted, correcting or replacing the erroneously recogn 
ized character with a selected candidate character, and displaying the corrected c 
5 haracter on the third display area; and 

storing the item and character data displayed on the third display area whe 
n a "CONFIRM 55 key is inputted. 

14. The method as set forth in claim 13, wherein the step of correcting the 
erroneously recognized character further comprises the steps of: 

10 when the "CORRECT 55 key is inputted, displaying a handwritten character 

recognition window on the second display area; 

when a handwritten character is inputted into the handwritten character re 
cognition window, recognizing the inputted handwritten character; and 

correcting or replacing the erroneously recognized character with the reco 
1 5 gnized character based on the handwritten character. 

15. The method as set forth in claim 13, wherein the step of correcting the 
erroneously recognized character further comprises the steps of: 

when the "CORRECT 55 key is inputted, driving a speech recognizer; 
allowing the speech recognizer to recognize an inputted speech signal and 
20 to convert the recognized speech signal into character data; and 

correcting or replacing the erroneously recognized character with the char 
acter data based on the recognized speech signal. 
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